Eliminating Incoherence from Subjective Estimates of Chance

Human judgment is an essential source of Bayesian probabilities but is plagued by incoherence when complex or conditional events are involved. We consider a method for adjusting estimates of chance over Boolean events so as to render them probabilistically coherent. The method works by searching for a sparse distribution that approximates a target set of judgments. (We show that sparse distributions suffice for this purpose.) The feasibility of our method was tested by randomly generating sets of coherent and incoherent estimates of chance over 30 to 50 variables. Even with 50 variables, good approximations were computed within a few hours. Empirical test was provided by asking people to estimate the chances of events relating to the stock market. The estimates of each participant were incoherent but well approximated by the coherent distribution constructed for him. In addition, the quadratic scores of the reconstructed estimates were reliably superior to the scores of the original estimates. In this sense, our correction method improves the objective accuracy of the judgments that it renders coherent. The judgments of all participants were then pooled to form an ”aggregate judge,” and a coherent approximation to these 1426 aggregated judgments was computed. The quadratic scores for the resulting (coherent) probabilities were superior to the original estimates and also to the coherent approximations computed for subjects individually. Our method thus offers a new approach to aggregating opinions, an important problem in decision theory.